Complex event recognition using constrained low-rank representation

a r t i c l e i n f o Complex event recognition is the problem of recognizing events in long and unconstrained videos. In this extremely challenging task, concepts have recently shown a promising direction where core low-level events (referred to as concepts) are annotated and modeled using a portion of the training data, then each complex event is described using concept scores, which are features representing the occurrence confidence for the concepts in the event. However, because of the complex nature of the videos, both the concept models and the corresponding concept scores are significantly noisy. In order to address this problem, we propose a novel low-rank formulation, which combines the precisely annotated videos used to train the concepts, with the rich concept scores. Our approach finds a new representation for each event, which is not only low-rank, but also constrained to adhere to the concept annotation, thus suppressing the noise, and maintaining a consistent occurrence of the concepts in each event. Extensive experiments on large scale real world dataset TRECVID Multimedia Event Detection 2011 and 2012 demonstrate that our approach consistently improves the discriminativity of the concept scores by a significant margin. The increasing popularity of digital cameras has been creating a tremendous growth in social media websites like YouTube. Along with the increased number of user-uploaded videos, the need to automatically detect and recognize the type of activities occurring in these videos has become crucial. However, in such unconstrained videos, automatic content understanding is a very challenging task due to the large intra-class variation, dynamic and heterogeneous background, and different capturing conditions. Therefore, this problem has recently gained significant attention. Most activity recognition methods are developed for constrained and short videos (5–10 s) as in [3–6]. These videos contain simple and well-defined human actions such as waving, running, and jumping. In contrast, in this paper, we consider more practical videos with realistic events, complicated contents, and significantly variable lengths. Refer to Fig. 2. The standard activity recognition methods do not incorporate evidences for detecting a particular action/event when deal with such unconstrained videos. To this end, the most recent approaches resorted to using low-level events called " concepts " as an intermediate representation [2,1]. In that, a complex event is described using concept scores, which is the occurrence confidence for the concepts in the video. For example, the event Birthday Party can be described as …